Moving image extracting apparatus, program and moving image extracting method

ABSTRACT

There is provided a moving image extracting apparatus including a blur value obtaining unit to obtain a blur value which indicates a blur degree of each frame constituting a moving image, a segment determining unit to discriminate the moving image between a stable segment of which variance of the blur values obtained by the blur value obtaining unit is lower than a first value and an unstable segment which is not the stable segment, and an extracting unit to perform segment extraction from the moving image based on the stable segment or the unstable segment obtained by the segment determining unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 12/796,749,filed Jun. 9, 2010, and claims the benefit of priority under 35 U.S.C.§119 from Japanese Patent Application No. 2009-167909, filed Jul. 16,2009, the entire contents of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving image extracting apparatus, aprogram and a moving image extracting method.

2. Description of the Related Art

Recently, an imaging device capable of taking a moving image byconverting incident light into electrical signals has been widelyspread. With this imaging device, a user can keep scenes of an athleticfestival of children, travel scenes and scenes of a pet as a movingimage.

Further, software for automatic editing to prepare a short movie byextracting a segment from a moving image obtained by the imaging devicehas been proposed. With this software for automatic editing, anextraction segment of the moving image is determined at random, forexample. An example of such software for automatic editing of a movingimage has been disclosed in Japanese Patent Application Laid-Open No.2004-159331.

SUMMARY OF THE INVENTION

However, with the above software for automatic editing, since theextraction segment of the moving image is determined at random, forexample, it has been difficult to extract a desired segment for a user.For example, there may be a case that the above software for automaticediting extracts a defocused and blurred part of the moving image evenwhen the user desires to extract an attractive part of the moving image.

In light of the foregoing, it is desirable to provide a novel andimproved moving image extracting apparatus, a program and a moving imageextracting method capable of performing segment extraction from a movingimage in accordance with a blur value of each frame.

According to an embodiment of the present invention, there is provided amoving image extracting apparatus including a blur value obtaining unitto obtain a blur value which indicates a blur degree of each frameconstituting a moving image, a segment determining unit to discriminatethe moving image between a stable segment of which variance of the blurvalues obtained by the blur value obtaining unit is lower than a firstvalue and an unstable segment which is not the stable segment and anextracting unit to perform segment extraction from the moving imagebased on the stable segment or the unstable segment obtained by thesegment determining unit.

The segment determining unit may include a primary processing unit whichsequentially selects a different frame in the moving image as a startpoint and defines a stable segment while expanding a segment includingthe selected frame until variance of the blur values exceeds the firstvalue.

The primary processing unit may repeat defining the stable segment untilany frame which is not defined as a stable segment and which is notselected as a start point does not exist.

The segment determining unit may include a secondary processing unitwhich determines whether or not variance of blur values of a segmentincluding each unstable segment and defined stable segments at beforeand after the unstable segment is smaller than the first value anddefines the segment as a stable segment when the variance of the blurvalues of the segment is smaller than the first value.

The segment determining unit may include a tertiary processing unitwhich expands the stable segment defined by the secondary processingunit until the variance of the blur values exceeds the first value.

The extracting unit may extract a segment of which average value of theblur values is smaller than a second value among the stable segments andthe unstable segments.

The extracting unit may extract a segment of which segment length islonger than a third value in addition to being smaller of the averagevalue of the blur values than the second value.

The extracting unit may extract a segment of which maximum blur value islarger than a fourth value and a segment of which segment length isshorter than a fifth value among the stable segments and the unstablesegments.

The extracting unit may further extract an unstable segment adjacent toa stable segment of which maximum blur value is larger than the fourthvalue.

The moving image extracting apparatus may further include a buffer totemporarily store a blur value of a new frame obtained by the blur valueobtaining unit. The segment determining unit may perform discriminatingof the moving image after blur values of unprocessed frames of apredetermined amount or more are stored at the buffer.

The segment determining unit may expand a stable segment including aframe immediately previous to an unprocessed frame to the unprocessedframe side until the variance of the blur values exceeds the first valuein a case that the immediately previous frame is included in the stablesegment.

According to another embodiment of the present invention, there isprovided a program for causing a computer to function as a blur valueobtaining unit to obtain a blur value which indicates a blur degree ofeach frame constituting a moving image, a segment determining unit todiscriminate the moving image between a stable segment of which varianceof the blur values obtained by the blur value obtaining unit is lowerthan a first value and an unstable segment which is not the stablesegment, and an extracting unit to perform segment extraction from themoving image based on the stable segment or the unstable segmentobtained by the segment determining unit.

According to another embodiment of the present invention, there isprovided a moving image extracting method, comprising the steps ofobtaining a blur value which indicates a blur degree of each frameconstituting a moving image discriminating the moving image between astable segment of which variance of the blur values is smaller than afirst value and an unstable segment which is not the stable segment, andperforming segment extraction from the moving image based on the stablesegment or the unstable segment.

According to the moving image extracting apparatus, the program and themoving image extracting method of the present invention described above,segment extraction from a moving image can be performed in accordancewith a blur value of each frame.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory view which illustrates an image extractingsystem according to an embodiment of the present invention;

FIG. 2 is an explanatory view which illustrates a schematic flow ofoperation of the moving image extracting apparatus;

FIG. 3 is a block diagram which illustrates a hardware configuration ofthe moving image extracting apparatus;

FIG. 4 is a functional block diagram which illustrates the configurationof the moving image extracting apparatus according to a firstembodiment;

FIG. 5 is an explanatory view which illustrates a specific example of astable segment and an unstable segment;

FIG. 6 is an explanatory view which illustrates a specific example of aprimary process by a primary processing unit;

FIG. 7 is an explanatory view which illustrates a specific example of asecondary process by a secondary processing unit;

FIG. 8 is an explanatory view which illustrates a specific example of atertiary process by a tertiary processing unit;

FIG. 9 is an explanatory view which illustrates an example of blurstatistics;

FIG. 10 is an explanatory view which illustrates a specific example of afailed cut extracted by an extracting unit;

FIG. 11 is an explanatory view which illustrates a specific example of asuggested cut extracted by the extracting unit;

FIG. 12 is a flowchart which describes a flow of operation of the movingimage extracting apparatus according to the first embodiment;

FIG. 13 is a functional block diagram which illustrates theconfiguration of a moving image extracting apparatus according to asecond embodiment;

FIG. 14 is an explanatory view which schematically illustrates a blurvalue of an unprocessed frame and a blur value of an unprocessed frame;

FIG. 15 is an explanatory view which schematically illustrates a blurvalue of an unprocessed frame and a blur value of an unprocessed frame;

FIG. 16 is a flowchart which describes a flow of segment determinationin the second embodiment;

FIG. 17 is a flowchart which describes the detailed flow of buffering;

FIG. 18 is a flowchart which describes the detailed flow ofpreprocessing; and

FIG. 19 is an explanatory view which illustrates a specific example ofthe preprocessing.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Embodiments of the present invention will be described in the followingorder.

1. Outline of image extracting apparatus2. First embodiment

2-1. Hardware configuration of image extracting apparatus

2-2. Functions of image extracting apparatus

2-3. Operation of image extracting apparatus

3. Second embodiment

4. Summary 1. Outline of Image Extracting Apparatus

First, a moving image extracting apparatus 20 according to an embodimentof the present invention will be schematically described with referenceto FIGS. 1 and 2.

FIG. 1 is an explanatory view of an image extracting system 1 accordingto an embodiment of the present invention. As illustrated in FIG. 1, theimage extracting system 1 includes an imaging device 10 and the movingimage extracting apparatus 20.

The imaging device 10 obtains a moving image including a plurality offrames by converting incident light into electrical signals. Further,the imaging device 10 stores the obtained moving image to a storagemedium mounted integrally on the imaging device 10 or a storage mediumattached to the imaging device 10 in a detachably attachable manner.Here, the imaging device 10 may start obtaining a moving image based onrecording start operation of a user and may end obtaining the movingimage based on recording end operation of the user.

The moving image extracting apparatus 20 obtains the moving image takenby imaging with the imaging device 10. For example, the moving imageextracting apparatus 20 may have a storage medium having the movingimage recorded and may obtain the moving image from the storage medium.Instead, the imaging device 10 may transmit a moving image by wired orwireless and the moving image extracting apparatus 20 may receive themoving image transmitted from the imaging device 10.

In FIG. 1, a personal computer (PC) is illustrated as an example of themoving image extracting apparatus 20. However, the moving imageextracting apparatus 20 is not limited to a PC. For example, the movingimage extracting apparatus 20 may be an information processing apparatussuch as a home video processing apparatus (a DVD recorder, avideocassette recorder and the like), a personal digital assistant(PDA), home game equipment and a home electrical appliance. Further, themoving image extracting apparatus 20 may be an information processingapparatus such as a cellular phone, a personal handyphone system (PHS),a portable music player, a portable video processing device and portablegame equipment.

The moving image extracting apparatus 20 extracts a segment from theobtained moving image. For example, the moving image extractingapparatus 20 extracts a failed cut or a suggested cut from the movingimage based on a blur degree of each frame which constitutes the movingimage. In the following, the flow until the moving image extractingapparatus 20 extracts a failed cut or a suggested cut is brieflydescribed with reference to FIG. 2.

FIG. 2 is an explanatory view illustrating a schematic flow of theoperation of the moving image extracting apparatus 20. As illustrated inFIG. 2, the moving image extracting apparatus 20 calculates a blur valueof each frame constituting a moving image (S40). Here, the blur valuebeing an index to indicate a blur degree indicates that the blur degreeis to be high as the value increases.

Next, the moving image extracting apparatus 20 determines a stablesegment and an unstable segment in the moving image (S42). Here, thestable segment is a segment having a blur value variance equal to orsmaller than a threshold value Th1. The unstable segment is a segmenthaving a blur value variance larger than the threshold value Th1, thatis, a segment having larger blur degree variation than that of thestable segment.

Next, the moving image extracting apparatus 20 determines an extractionsegment based on a list indicating blur statistics of each segment(S44). For example, the moving image extracting apparatus 20 may extracta suggested cut or a failed cut based on the list indicating blurstatistics such as an average blur value, a maximum blur value and ablur value variance of each segment.

As described above, the moving image extracting apparatus 20 accordingto the embodiment of the present invention is capable of extracting asuggested cut or a failed cut based on a blur value of each frameconstituting a moving image. In the following, the moving imageextracting apparatus 20 will be described in detail with reference toFIGS. 13 to 19.

2. First Embodiment (2-1. Hardware Configuration of Image ExtractingApparatus)

FIG. 3 is a block diagram illustrating the hardware configuration of themoving image extracting apparatus 20. The moving image extractingapparatus 20 includes a central processing unit (CPU) 201, a read onlymemory (ROM) 202, a random access memory (RAM) 203 and a host bus 204.In addition, the moving image extracting apparatus 20 includes a bridge205, an external bus 206, an interface 207, an input device 208, anoutput device 210, a storage device (HDD) 211, a drive 212 and acommunication device 215.

The CPU 201 functions as an arithmetic processing unit and a controllingunit and controls general operation in the moving image extractingapparatus 20 in accordance with a variety of programs. The CPU 201 maybe a microprocessor. The ROM 202 stores the programs and arithmeticparameters to be used by the CPU 201. The RAM 203 temporarily storesprograms to be used during the operation of the CPU 201, parameters tovary appropriately during the operation thereof and the like. These aremutually connected by the host bus 204 constituted with a CPU bus andthe like.

The host bus 204 is connected to the external bus 206 such as aperipheral component interconnect/interface (PCI) bus via the bridge205. Here, it is not necessary to separately constitute the host bus204, the bridge 205 and the external bus 206. The functions thereof maybe mounted on a single bus.

The input device 208 is constituted with an input means such as a mouse,a keyboard, a touch panel, a button, a microphone, a switch and a leverto input information by a user, and an input controlling circuit togenerate an input signal based on the input by the user and to outputthe signal to the CPU 201. The user of the moving image extractingapparatus 20 can input a variety of data and instruct process operationby operating the input device 208.

The output device 210 includes a display device such as a cathode raytube (CRT) display device, a liquid crystal display (LCD) device, anorganic light emitting diode (OLED) device and a lamp. Further, theoutput device 210 includes an audio output device such as a speaker anda headphone. The output device 210 outputs a reproduced content, forexample. Specifically, the display device displays various types ofinformation such as reproduced video data with texts or images.Meanwhile, the audio output device converts reproduced audio data andthe like into audio and outputs the audio.

The storage device 211 is a device for data storage configured to be anexample of a memory unit of the moving image extracting apparatus 20according to the present embodiment. The storage device 211 may includea storage medium, a recording device to record data at the storagemedium, a reading device to read the data from the storage medium, and adeleting device to delete the data recorded at the storage medium. Thestorage device 211 is configured with a hard disk drive (HDD), forexample. The storage device 211 drives the hard disk and stores programsto be executed by the CPU 201 and a variety of data.

The drive 212 is a reader/writer for the storage medium and isincorporated by or externally attached to the moving image extractingapparatus 20. The drive 212 reads the information stored at a mountedremoval storage medium 24 such as a magnetic disk, an optical disk, amagneto-optical disk and a semiconductor memory and outputs theinformation to the RAM 203.

The communication device 215 is a communication interface constitutedwith a communication device and the like to be connected to acommunication network 12, for example. Here, the communication device215 may be a wireless local area network (LAN) compatible communicationdevice, a wireless USB compatible communication device or a wiredcommunication device to perform communication with a cable.

(2-2. Functions of Image Extracting Apparatus)

In the above, the hardware configuration of the moving image extractingapparatus 20 is described with reference to FIG. 3. Next, the functionsof the moving image extracting apparatus 20 will be described withreference to FIGS. 4 to 11.

FIG. 4 is a functional block diagram illustrating the configuration ofthe moving image extracting apparatus 20 according to the firstembodiment. As illustrate in FIG. 4, the moving image extractingapparatus 20 according to the first embodiment includes a memory unit216, a blur value obtaining unit 218, a segment determining unit 220, astatistics obtaining unit 230 and an extracting unit 234.

The memory unit 216 is a storage medium having a moving image storedfrom the imaging device 10. A non-volatile memory, a magnetic disk, anoptical disk, and a magneto-optical (MO) disk may be adopted as thestorage medium. For example, an electrically erasable programmableread-only memory (EEPROM) and an erasable programmable read-only memory(EPROM) may be adopted as the non-volatile memory. A hard disk and adiscoidal magnetic disk may be adopted as the magnetic disk. Further, acompact disc (CD), a digital versatile disc recordable (DVD-R) and ablu-ray disc (BD, a registered trademark) may be adopted as the opticaldisk. Here, the memory unit 216 may be a storage medium receiving amoving image transmitted from the imaging device 10.

The blur value obtaining unit 218 obtains a blur value of each frameconstituting the moving image stored at the memory unit 216. The blurvalue can be obtained based on an edge part of a flame. For example, themore a frame is defocused and blurred, the more gradual the variation ofbrightness distribution at an edge part thereof is. Then, the more aframe is focused and not blurred, the sharper the variation ofbrightness distribution at an edge part thereof is. Accordingly, theblur value obtaining unit 218 may obtain a blur value based on variationsharpness of brightness distribution at an edge part thereof.

A segment determining unit 220 discriminates the moving image between astable segment having a blur value variance equal to or smaller than athreshold value Th1 (i.e., a first value) and an unstable segment havinga blur value variance larger than the threshold value Th1 based on theblur value of each flame obtained by the blur value obtaining unit 218,as illustrated in FIG. 5.

FIG. 5 is an explanatory view illustrating a specific example of stablesegments and unstable segments. As illustrated in FIG. 5, the segmentdetermining unit 220 determines a segment having a small blur valuevariation as a stable segment (i.e., a solid-line oval part) anddetermines a segment having a large blur value variation as an unstablesegment (i.e., a dashed-line oval part). In order to perform suchsegment determination, the segment determining unit 220 includes aprimary processing unit 221, a secondary processing unit 222 and atertiary processing unit 223.

The primary processing unit 221 sequentially selects a different framein the moving image as a start point and defines a stable segment whileexpanding the segment including the selected frame until the blur valuevariance exceeds the threshold value Th1. More specifically, the primaryprocessing unit 221 defines the stable segment while expanding thesegment as the selected frame being the start point in the futuredirection until the blur value variance exceeds the threshold value Th1,and then, expanding the segment in the past direction as well until theblur value variance exceeds the threshold value Th1. In the following,detailed description will be made with reference to FIG. 6.

FIG. 6 is an explanatory view illustrating a specific example of theprimary process by the primary processing unit 221. As illustrated atthe first row in FIG. 6, all frames are candidate unstable segments atthe beginning. In FIG. 6, “X−8”, . . . , “X”, . . . , “X+7” denote framenumbers.

Here, the second row in FIG. 6 illustrates a case that the primaryprocessing unit 221 selects the frame “X” as the start point. In thiscase, the primary processing unit 221 expands the segment in the futuredirection to the frames “X+1”, “X+2”, “X+3” having the frame “X” as thestart point. Since the blur value variance exceeds the threshold valueTh1 when expanding the segment in the future direction to the frame“X+4”, the primary processing unit 221 stops expanding the segment inthe future direction at the frame “X+3” as illustrates at the third rowin FIG. 6.

Subsequently, the primary processing unit 221 expands the segment in thepast direction to the frames “X−1”, “X−2” having the frame “X” as thestart point. Since the blur value variance exceeds the threshold valueTh1 when expanding the segment in the past direction to the frame “X−3”,the primary processing unit 221 stops expanding the segment in the pastdirection at the frame “X−2” as illustrated at the fourth row in FIG. 6.By expanding the segment as described above, the primary processing unit221 defines the stable segment.

Then, the primary processing unit 221 repeats the primary process ofFIG. 6 as selecting a frame among the candidate unstable segments whichare not defined as stable segments. For example, the primary processingunit 221 may select a frame having the maximum blur value or a framehaving the minimum blur value among the candidate unstable segments. Theprimary processing unit 221 repeats the primary process until all of theframes are included in the stable segment or are selected respectivelyas the start point.

The secondary processing unit 222 determines whether or not the blurvalue variance of the segment including each candidate unstable segment(each primary instable segment) and the defined stable segments at bothbefore and after the candidate unstable segment is equal to or smallerthan the threshold value Th1. When the blur value variance of the abovesegment is equal to or smaller than the threshold value Th1, thesecondary processing unit 222 defines the entire above segment as astable segment. In the following, the secondary process by the secondaryprocessing unit 222 will be specifically described with reference toFIG. 7.

FIG. 7 is an explanatory view illustrating a specific example of thesecondary process by the secondary processing unit 222. Here,color-filled parts at the upper row in FIG. 7 denote segments defined asstable segments by the first processing unit 221 and the other partsdenote candidate unstable segments which are not defined as stablesegments by the first processing unit 221. For example, the segments A,C, E and G at the upper row in FIG. 7 are stable segments and thesegments B, D and F are candidate unstable segments.

Here, the secondary processing unit 222 selects the segment B being acandidate unstable segment and the segments A and C being stablesegments at both before and after the segment B. Then, the secondaryprocessing unit 222 calculates the blur value variance of selectedsegment of A, B and C. When the calculated blur value variance is equalto or smaller than the threshold value Th1, the entire segment of A, Band C is redefined as a stable segment as illustrated at the lower rowin FIG. 7.

Subsequently, the secondary processing unit 222 selects the segment Dbeing a candidate unstable segment and the segment E and the redefinedsegment H being stable segments. Then, the secondary processing unit 222performs the secondary process as similar to the above. When the blurvalue variance of the segment of H, D and E exceeds the threshold valueTh1, the states of respective segments H, D and E are maintained. Thesecondary processing unit 222 repeats the secondary process indicated inFIG. 7 until any segment possibly to be redefined as a stable segmentdoes not exist.

The tertiary processing unit 223 expands the stable segment defined bythe first processing unit 221 and the second processing unit 222 untilthe blur value variance exceeds the threshold value Th1. This is becausethe stable segment may be assumed to be further expanded as the blurvalue variance of the stable segment is decreased due to uniting of aplurality of segments with the secondary process by the secondaryprocessing unit 222. In the following, the tertiary process by thetertiary processing unit 223 will be specifically described withreference to FIG. 8.

FIG. 8 is an explanatory view illustrating a specific example of thetertiary process by the tertiary processing unit 223. Here, the segmentH at the upper row in FIG. 8 is the segment redefined as the stablesegment by the secondary processing unit 222.

As illustrated at the lower row in FIG. 8, the tertiary processing unit223 defines a segment H′ as expanding the segment H until the blur valuevariance exceeds the threshold value Th1. FIG. 8 exemplifies an examplethat the tertiary processing unit 223 expands the segment H in thefuture direction. However, the tertiary processing unit 223 can expandthe segment H in the past direction as well. The tertiary processingunit 223 performs the tertiary process indicated in FIG. 8 on everyboundary between each stable segment and each unstable segment.

As described above, the segment determining unit 220 performs theprimary to tertiary processes. Consequently, the segment determiningunit 220 determines the stable segments defined with the tertiaryprocess by the tertiary processing unit 223 as stable segments anddetermines the rest of segments as unstable segments.

The statistics obtaining unit 230 obtains the respective blur statisticslists of each stable segment and each unstable segment determined by thesegment determining unit 220.

FIG. 9 is an explanatory view illustrating an example of blurstatistics. As illustrated in FIG. 9, the blur statistics includes asegment ID, a start frame number, an end frame number, an average blurvalue, a maximum blur value, a minimum blur value, a blur value varianceand the like.

The extracting unit 234 extracts a failed cut, a suggested cut (i.e., asuccessful cut) or the like from the moving image based on the blurstatistics of each segment obtained by the statistics obtaining unit230. In the following, criteria for extracting a failed cut and criteriafor extracting a suggested cut will be described.

(Extraction of Failed Cut)

For example, the extracting unit 234 extracts a segment corresponding toany one of following criteria as a failed cut.

(1) A segment of which maximum blur value is larger than a thresholdvalue Th4 (i.e., a fourth value)(2) An unstable segment before or after a stable segment of whichmaximum blur value is larger than the threshold value Th4 (i.e., thefourth value)(3) A segment of which segment length is shorter than a threshold valueTh5 (i.e., a fifth value)

In the following, a specific example of a failed cut to be extracted bythe extracting unit 234 will be described with reference to FIG. 10.

FIG. 10 is an explanatory view illustrating the specific example of thefailed cut to be extracted by the extracting unit 234. In FIG. 10,color-filled segments O, Q, S, U and W denote stable segments and othersegments P, R, T and V denote unstable segments. The respective maximumblur values of the segments O, S, and W are larger than the thresholdvalue Th4 and the respective maximum blur values of the segments Q and Uare smaller than the threshold value Th4.

In this case, the extracting unit 234 extracts the stable segments Q andU as failed cuts based on the above (1). Further, the extracting unit234 extracts the unstable segments P, R, T and V as failed cuts based onthe above (2). In addition, the extracting unit 234 extracts the stablesegment S of which segment length is shorter than the threshold valueTh5 as a failed cut based on the above (3). As a result, the segments Pto V are extracted as a failed cut in the example of FIG. 10.

For example, by deleting the failed cut extracted as described abovefrom the moving image, a user can enhance density of worthwhile parts towatch and listen in the moving image. Here, the extraction criteriadescribed in the above (1) to (3) are just examples. A criterion may benewly added and any of the criteria may be deleted.

(Extraction of Suggested Cut)

For example, the extracting unit 234 extracts a segment corresponding toall of the following criteria as a suggested cut.

(4) A segment of which average blur value is equal to or smaller than athreshold value Th2 (i.e., a second value)(5) A segment of which segment length is equal to or longer than athreshold value Th3 (i.e., a third value)(6) A segment of which blur value variance is equal to or smaller than athreshold value Th6 (i.e., a sixth value)

In the following, a specific example of a suggested cut to be extractedby the extracting unit 234 will be described with reference to FIG. 11.

FIG. 11 is an explanatory view illustrating the specific example of thesuggested cut to be extracted by the extracting unit 234. As illustratedin FIG. 11, the average blur value of the stable segment X is smallerthan the threshold value Th2, so that the above (4) is satisfied.Further, the blur value variance of the stable segment X is smaller thanthe threshold value Th6, so that the above (6) is satisfied.Accordingly, when the segment length of the stable segment X is equal toor longer than the threshold value Th3 to satisfy the above (5) as well,the extracting unit 234 extracts the stable segment X as a suggestedcut.

The moving image extracting apparatus 20 provides the suggested cutextracted as described above to a user, for example. Accordingly, it ispossible for the user to selectively watch and listen a worthwhile partto watch and listen which is not defocused in the moving image. Here,the extraction criteria as described in the above (4) to (6) are justexamples. A criterion may be newly added and any of the criteria may bedeleted.

(2-3. Operation of Image Extracting Apparatus)

In the above, the configuration and functions of the moving imageextracting apparatus 20 according to the first embodiment of the presentinvention have been described. Next, operation of the moving imageextracting apparatus 20 according to the first embodiment will bedescribed with reference to FIG. 12.

FIG. 12 is a flowchart describing the operational flow of the movingimage extracting apparatus 20 according to the first embodiment. In FIG.12, the flow of the process of S42 in FIG. 2 is described in detail.

As illustrated in FIG. 12, the moving image extracting apparatus 20determines the unstable segment (S4) by performing the primary process(S 1), the secondary process (S2) and the tertiary process (S3) afterthe blur value of each frame is obtained by the blur value obtainingunit 218.

(Primary Process)

The primary processing unit 221 repeats the primary process until all ofthe frames are evaluated (S1-1). Specifically, the primary processingunit 221 selects a frame as the start point among unevaluated frames inthe moving image (S1-2). Here, the unevaluated frame denotes a framewhich is not defined as a stable segment and which has not been selectedas the start point.

Then, the primary processing unit 221 expands the segment in the futuredirection having the selected frame as the start point until the blurvalue variance exceeds the threshold value Th1 (S1-3). Then, the primaryprocessing unit 221 defines the stable segment by expanding the segmentin the past direction as well having the selected frame as the startpoint until the blur value variance exceeds the threshold value Th1 (S1-4).

(Secondary Process)

The secondary processing unit 222 repeats the secondary process untilall of the candidate unstable segments are processed (S2-1).Specifically, the secondary processing unit 222 selects a stable segmentwhich is adjacent to an unprocessed unstable segment (S2-2). Then, thesecondary processing unit 222 determines whether or not the blur valuevariance of the entire segment of the selected three or two segments isequal to or smaller than the threshold value Th1 (S2-3).

When the blur value variance of the above entire segment exceeds thethreshold value Th1, the secondary processing unit 222 puts thecandidate unstable segment selected in S2-2 into “processed” (S2-4). Onthe other hand, when the blur value variance of the above entire segmentis equal to or smaller than the threshold value Th1, the secondaryprocessing unit 222 unites and redefines the above entire segment as astable segment (S2-5).

(Tertiary Process)

The tertiary processing unit 223 repeats the tertiary process untilevery boundary between each stable segment and each candidate unstablesegment are processed (S3-1). Specifically, the tertiary processing unit223 selects an unprocessed boundary (S3-2) and expands the stablesegment by one frame until the blur value variance exceeds the thresholdvalue Th1 (S3-3).

With the above primary to tertiary processes, the stable segmentsdefined with the tertiary process by the tertiary processing unit 223are determined to be stable segments and the other segments aredetermined to be unstable segments (S4). Consequently, the extractingunit 234 is to be capable of extracting a failed cut or a suggested cutin accordance with a purpose or a user instruction.

3. Second Embodiment

Next, a moving image extracting apparatus 20′ according to the secondembodiment of the present invention will be described with reference toFIGS. 13 to 19. Here, the imaging device 10 illustrated in FIG. 1 isassumed to be the moving image extracting apparatus 20′, for example.The moving image extracting apparatus 20′ is capable of determining astable segment or an unstable segment during the imaging is performed.

FIG. 13 is a functional block diagram illustrating the configuration ofthe moving image extracting apparatus 20′ according to the secondembodiment. As illustrated in FIG. 13, the moving image extractingapparatus 20′ according to the second embodiment includes the blur valueobtaining unit 218, a segment determining unit 220′ the statisticsobtaining unit 230, the extracting unit 234, an imaging unit 238 and abuffer 242.

The imaging unit 238 sequentially obtains new frames by imaging. Theblur value obtaining unit 218 obtains a blur value of the frame obtainedby the imaging unit 238. The buffer 242 sequentially and temporarilystores the blur values obtained by the blur value obtaining unit 218.

The segment determining unit 220′ includes the primary processing unit221, the secondary processing unit 222, the tertiary processing unit 223and a preprocessing unit 250. In the following, blur values ofunprocessed frames and blur values of processed frames will be describedwith reference to FIGS. 14 and 15.

FIGS. 14 and 15 are explanatory views schematically illustrating theblur values of the unprocessed frames and blur values of the processedframes. As illustrated in FIG. 14, the segment determining unit 220′ iscapable of referring to the blur values of the frames temporarily storedat the buffer 242 in the moving image. Here, as illustrated in FIG. 15,the present embodiment will be described as the frame number of thenewest blur value being “0”, the oldest unprocessed frame number being“−L”, and the oldest processed frame number being “−N” which aretemporarily stored at the buffer 242.

FIG. 16 is a flowchart describing the flow of segment determinationaccording to the second embodiment. As illustrated in FIG. 16, first,buffering is performed on the blur value at the buffer 242 (S5). Then,the preprocessing unit 250 of the segment determining unit 220′ performspreprocessing (S6) and the primary to tertiary processing units 221 to223 respectively performs the primary to tertiary processes (S7). In thefollowing, the detailed flow of the buffering (S5) will be describedwith reference to FIG. 17 and the detailed flow of the preprocessing(S6) will be described with reference to FIGS. 18 and 19. Since theprimary to tertiary processes (S7) are the same as described in thefirst embodiment, the description will not be repeated in the presentembodiment.

FIG. 17 is a flowchart illustrating the detailed flow of the buffering.First, the segment determining unit 220′ determines whether or not theblur values of the unprocessed frames being buffered at the buffer 242are less than those of L frames (S5-1). In the case that the blur valuesof the unprocessed frames being buffered are equal to or more than thoseof L frames, the preprocessing unit 250 of the segment determining unit220′ performs the preprocessing (S6).

On the other hand, in the case that the blur values of the unprocessedframes being buffered are less than those of L frames, the process fromS5-1 is repeated when the last frame is not included in the frames beingbuffered (S5-2). When the last frame is included in the frames beingbuffered, the preprocessing unit 250 of the segment determining unit220′ performs the preprocessing (S6) unless the blur value of the lastframe has been processed (S5-3). On the other hand, when blur value ofthe last frame has been processed, the segment determining process ends.

FIG. 18 is a flowchart describing the detailed flow of thepreprocessing. As illustrated in FIG. 18, the preprocessing unit 250determines whether or not the frame (L−1) is included in the stablesegment (S6-1) after the buffering (S5). When the frame (L−1) isincluded in the stable segment, the stable segment including the frame(L−1) is expanded to the frame (L) (S6-2). Then, when the blur valuevariance of the segment expanded to the frame (L) is smaller than thethreshold value Th1 (S6-3), the preprocessing unit 250 redefines thesegment expanded to the frame (L) as a stable segment (S6-4).Thereafter, the processes from the buffering (S5) are repeated. On theother hand, when the frame (L−1) is not included in the stable segment(S6-1) or when the blur value variance of the segment expanded to theframe (L) is equal to or larger than the threshold value Th1 (S6-3), theprimary to tertiary processes are performed on the referable blur valuesbeing buffered (S7). In the following, the above preprocessing will bespecifically described with reference to FIG. 19.

FIG. 19 is an explanatory view illustrating a specific example of thepreprocessing. As illustrated in FIG. 19, when the frame just before theunprocessed frame (−L) is in a stable segment, the preprocessing unit250 determines whether or not the blur value variance after adding theframe (−L) to the above stable segment is to be equal to or larger thanthe threshold value Th1. When the blur value variance is not to be equalto or larger than the threshold value Th1, the preprocessing unit 250redefines the stable segment to include the frame (−L) as illustrated atthe lower row in FIG. 19. On the other hand, when the blur valuevariance is to be equal to or larger than the threshold value Th1, theprimary to tertiary processes described in the first embodiment areperformed on the referable blur values being buffered. That is, thereferable blur values being buffered receive the primary process (i.e.,selecting the start point from unprocessed frames and expanding a stablesegment), the secondary process (i.e., uniting a stable segment, anunstable segment and another stable segment) and the tertiary process(i.e., moving a boundary).

4. Summary

As described above, according to the first embodiment of the presentinvention, it is possible to discriminate a moving image between astable segment and an unstable segment based on a blur value of eachframe and to extract a failed cut or a suggested cut by utilizing blurstatistics of each segment. Further, according to the second embodimentof the present invention, it is possible to determine a stable segmentand an unstable segment during imaging is performed.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

For example, each step in the processes of the moving image extractingapparatus 20 herein is not necessarily performed in time sequence inorder described in the flowchart. For example, each step of theprocesses of the moving image extracting apparatus 20 may include aprocess which is serially performed and a process which is separatelyperformed.

Further, it is also possible to prepare a computer program to exertfunctions similar to the hardware such as the CPU 201, the ROM 202 andthe RAM 203 incorporated in the moving image extracting apparatus 20.Here, a storage medium storing the computer program may be provided.Further, a series of processes can be performed with hardware byconstituting respective functional blocks in the functional blockdiagram in FIG. 4 with hardware.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2009-167909 filedin the Japan Patent Office on Jul. 16, 2009, the entire content of whichis hereby incorporated by reference.

What is claimed is:
 1. A moving image extracting apparatus comprising: ablur value obtaining unit to obtain a blur value which indicates a blurdegree of each frame constituting a moving image; a segment determiningunit to discriminate the moving image between a stable segment of whichvariance of the blur values obtained by the blur value obtaining unit islower than a first value and an unstable segment which is not the stablesegment; and an extracting unit to perform segment extraction from themoving image based on the stable segment or the unstable segmentobtained by the segment determining unit.
 2. The moving image extractingapparatus according to claim 1, wherein the segment determining unitincludes a primary processing unit which sequentially selects adifferent frame in the moving image as a start point and defines astable segment while expanding a segment including the selected frameuntil variance of the blur values exceeds the first value.
 3. The movingimage extracting apparatus according to claim 2, wherein the primaryprocessing unit repeats defining the stable segment until any framewhich is not defined as a stable segment and which is not selected as astart point does not exist.
 4. The moving image extracting apparatusaccording to claim 2, wherein the segment determining unit includes asecondary processing unit which determines whether or not variance ofblur values of a segment including each unstable segment and definedstable segments at before and after the unstable segment is smaller thanthe first value and defines the segment as a stable segment when thevariance of the blur values of the segment is smaller than the firstvalue.
 5. The moving image extracting apparatus according to claim 4,wherein the segment determining unit includes a tertiary processing unitwhich expands the stable segment defined by the secondary processingunit until the variance of the blur values exceeds the first value. 6.The moving image extracting apparatus according to claim 5, wherein theextracting unit extracts a segment of which average value of the blurvalues is smaller than a second value among the stable segments and theunstable segments.
 7. The moving image extracting apparatus according toclaim 6, wherein the extracting unit extracts a segment of which segmentlength is longer than a third value in addition to being smaller of theaverage value of the blur values than the second value.
 8. The movingimage extracting apparatus according to claim 1, wherein the extractingunit extracts a segment of which maximum blur value is larger than afourth value and a segment of which segment length is shorter than afifth value among the stable segments and the unstable segments.
 9. Themoving image extracting apparatus according to claim 8, wherein theextracting unit further extracts an unstable segment adjacent to astable segment of which maximum blur value is larger than the fourthvalue.
 10. The moving image extracting apparatus according to claim 1,further comprising: a buffer to temporarily store a blur value of a newframe obtained by the blur value obtaining unit; wherein the segmentdetermining unit performs discriminating of the moving image after blurvalues of unprocessed frames of a predetermined amount or more arestored at the buffer.
 11. The moving image extracting apparatusaccording to claim 10, wherein the segment determining unit expands astable segment including a frame immediately previous to an unprocessedframe to the unprocessed frame side until the variance of the blurvalues exceeds the first value in a case that the immediately previousframe is included in the stable segment.
 12. A program for causing acomputer to function as: a blur value obtaining unit to obtain a blurvalue which indicates a blur degree of each frame constituting a movingimage; a segment determining unit to discriminate the moving imagebetween a stable segment of which variance of the blur values obtainedby the blur value obtaining unit is lower than a first value and anunstable segment which is not the stable segment; and an extracting unitto perform segment extraction from the moving image based on the stablesegment or the unstable segment obtained by the segment determiningunit.
 13. A moving image extracting method, comprising the steps of:obtaining a blur value which indicates a blur degree of each frameconstituting a moving image; discriminating the moving image between astable segment of which variance of the blur values is smaller than afirst value and an unstable segment which is not the stable segment; andperforming segment extraction from the moving image based on the stablesegment or the unstable segment.